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Abstract. This paper is an initial report of the data analysis phase of a larger study 
that traces the evolution of Computer Assisted Language Learning (CALL). All 
published Research Articles (RAs) from four major CALL journals — ReCALL, 
CALL, Language Learning & Technology (LL&T) and CALICO Journal — from the 
very first issues to the end of 2019 were downloaded, sorted, and checked to form 
the final corpus of 426 highly cited RAs. The trends and themes (research contexts, 
research participants, and theoretical and methodological considerations of the 
RAs) were all recorded to see how CALL has evolved over time. Primary findings 
indicate that empirical studies where learners are physically or virtually involved 
in the process of technology-mediated language instruction dominate the field of 
CALL research. Authors resort to both quantitative and qualitative methodologies 
for data collection and analysis, though mixed-methods has gained more weight in 
the past two decades. Sociocultural theory stands over and above other theories in 
Second Language Acquisition (SLA) to frame CALL studies. The paper discusses 
these issues, and problems detected. 
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1. Introduction 


In the past five decades, CALL has been extensively used to support learning, 
evaluate learners’ language, and collect data for both in- and out-of-class inquiry. 
With the expansion of CALL, dedicated journals have emerged as technology- 
specialized platforms to invite their readers to keep up with the pace of technological 
advances through academic RAs. These CALL papers have been combined with 
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other academic genres such as book chapters, theses, and conference proceedings to 
form CALL history over time. This study, as a part of an ongoing PhD dissertation, 
traces that history by limiting its scope to highly cited CALL-oriented RAs. The 
question that stands out is how the contextual, theoretical and methodological 
dimensions of high-impact CALL RAs have changed over time. An ultimate aim of 
this narrative synthesis is the provision of a roadmap that can complement similar 
surveys (Gillespie, 2020; Lim & Aryadoust, 2021), tell us where we are now with 
CALL and how we got here — and, building on that, propose avenues for further 
research. 


2. Method 


The pool of data consists of a corpus of 2,397 RAs published in English, from the 
very first issues up to the end of 2019, in four major CALL journals: ReCALL, 
CALL, LL&T and CALICO Journal. Google Scholar citation metrics were used 
as a criterion to reduce the pool of papers and produce a manageable final corpus 
of 426 high-impact RAs (see Choubsaz, Boulton, & Jalilifar, 2020 for the choice 
of journals and data collection procedure). A coding book was developed to 
investigate CALL research trends and extract the main themes: research contexts, 
participants, and theoretical and methodological considerations. The first category 
covered countries, settings, programs, learning environments, and technology 
use, while the second recorded the status, age group, proficiency, L1, and L2 of 
the research participants. For the third category, theoretical and methodological 
considerations, our synthesis covered the applied methodologies of the studies and 
the theories/approaches the authors adopt to frame their work. About 10% of the 
data was coded in two separate rounds by two independent coders and an inter-rater 
r of 89.8 was attained. Part of the coding procedure was done manually, but our 
synthesis also made use of NVivo 12 Plus and AntConc in extracting, categorizing, 
and merging the principal themes and sub-themes. 


3. Results and discussion 


The United States is the leading contributor to the field (31% of all publications), 
English as a Second Language and English as a Foreign Language — ESL and 
EFL — (32%) are the most used settings, and a huge portion of studies are 
conducted in university contexts (58%) either in classrooms (28%) or laboratories 
(15%), with an increasing number of virtual learning environments (26%) in 
the past two decades. Research participants have the status of learners (68%) 
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who are predominantly at the intermediate (21%) and advanced (14%) levels of 
language proficiency and are aged 18-30 (20%). Nearly half of participants speak 
English (31%) or Chinese (14%) as their first languages; the target language 
is overwhelmingly English (42%). These findings no doubt partly reflect the 
reality of learners and teachers as well as research cultures (and the huge input 
from researchers in the US working in second language environments), though 
it should be remembered that the journals studied all publish in English, which 
may encourage studies on this language by researchers sufficiently familiar with 
it themselves. Another point, more difficult to quantify as rarely explicitly stated, 
is that many authors/researchers are also teachers in university settings, while 
few teachers are researchers worldwide; this raises questions about how research 
reflects typical teaching concerns and practices. 


Authors frequently adhere to quantitative methodologies (25%) for data collection 
and analysis, followed by qualitative (20%) and eclectic methodologies (18%) 
defined as a combination of qualitative and quantitative data and analysis with no 
mention of the term mixed-methods. In this, CALL research and SLA in general 
may be in thrall to Science, Technology, Engineering, and Mathematic (STEM) 
practices, with extensive use of statistical analysis (notably null hypothesis 
significance testing, despite the problems noted in journals such as Language 
Learning, and increasingly in psychology and medicine as a whole), partly 
reflected in the use of terms such as subject, treatment, and intervention (as if the 
learners were sick), while control groups are usually comparison groups, and so 
on. Power is limited by small sample size, generalizability by the use of intact 
groups in a single context, and impact by the lack of delayed data collection which 
is still vastly underused. More encouragingly, truly mixed-method studies (where 
the same data set is subjected to multiple analysis, rather than simply aligning e.g. 
post-tests and questionnaires) are on the increase (6%). 


The theoretical underpinnings the authors bring to the fore to justify their research 
are not specific to CALL but are SLA-oriented. Sociocultural theory is the most 
frequently cited (15%), followed by interactionism (9%), social-constructivism 
(8%) and collaborative learning (7%), with many others being cited just a few times. 
In line with the results of Yim and Warschauer (2017), our analysis indicates that 
120 RAs (28%), especially the ones published in the 1980’s, 1990’s, and 2000’s, 
failed to adopt a theory or an approach to frame their work. This lack of theoretical 
underpinning is perhaps to be expected in the empirical studies favored in RAs, 
where it often seems to frame a study almost as an afterthought rather than being 
a major factor inspiring the research itself. Pure theory/position papers are more 
likely found in books or chapters, or in conference presentations and proceedings, 


54 


Context, method, and theory in CALL research articles 


so there would seem to be a place for theory to drive empirical research or to be 
tested directly. 


An increased interest in theory alongside pedagogical considerations might also 
help reduce a tendency to seek technological novelty as the main thrust behind a 
publication. In our survey, text chatting and WeChat (7%), multimedia (5%), digital 
gaming (4%) and mobile phones (4%) seem to be used quite frequently, these low 
numbers highlighting the diversity of technology, perhaps for technology’s sake. 
In other words, research should not be driven by noticing (or creating) a new tool 
and seeing what it can be used for, but by learners’ needs, in addition to research 
gaps. Contrasting this is a possible lack of ambition, with many studies covering 
well-trod ground and reproducing (rather than replicating) earlier findings, perhaps 
due to the difficulty of gaining a solid overview of CALL research which is 
published in so many diverse sources. The ‘publish or perish’ syndrome is no doubt 
a contributing factor, with output motivated by career rather than new questions to 
be supported by original research. 


A final issue, not specific to CALL, is the lack of clarity while collecting 
demographic information of the RAs. In fact, not many papers report all the useful 
information, or if they do, they report it in ways that are not easy to compare. Even 
very basic information like the number of participants, age, L1 and L2, context, 
discipline, the technology used, and what the learners actually did is often missing, 
incomplete, or of limited use. Two particular problems here are duration and L2 
proficiency. The former is variably measured in minutes/hours, sessions, weeks, 
months, semesters, and years; but a ten-week course may be intensive and include 
out-of-class work, or involve a one-hour class every second week not including 
holidays, with only a fraction of the time given over to the main research question. 
The latter, if stated, may simply be listed as e.g. ‘intermediate’ with no further 
indication, which clearly means different things to different people. 


4. Conclusions 


As the study is still ongoing, definitive results are yet to be reached. However, 
the increasing number of high-impact publications (204 in the 1980’s to 893 in 
the 2010’s) and the preponderance of empirical studies (63%) with learners being 
physically or virtually present in classrooms, all attest to the healthy state of CALL 
research after four decades of presence. On the flip side, the increase in output 
has not resolved some long-standing controversies. This paper has outlined some 
of these, including methodological problems of design, logistical or contextual 
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constraints, a lack of theoretical consideration, and poor reporting practices. 
While many of these issues are long-standing, there is some indication of positive 
evolution, such as the increase of mixed-methods studies. 
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